1. Field of the Invention
The present invention relates to an image processing method, apparatus, and computer program product which compensate for correcting distortions in a scanned text image to enable and enhance character recognition thereof.
2. Description of the Related Art
Handheld scanners which manually scan images written on a hard-copy document are widely available today. A handheld scanner commonly found in the marketplace is of the rolling type, in which the user physically presses the scanning wand against the document and rolls the wand across the desired image area. A roller member is provided on the scanning wand adjacent the scanner window to physically contact the document when external pressure is applied and enable movement while maintaining contact. The images on the document are optically read as the scanner is moved, converted to image data, and uploaded to the host device to which the scanner is connected.
In order to obtain acceptable character recognition rates in rolling type handheld scanners, the roller member must be moved in a stable manner along the text and in continuous contact with the document original roller. Typically, this roller member reads image data 16 times for every 1 mm of travel. The width of the roller member is sized to provide a certain linearity to the scanning direction, thereby enabling best-scenario scanning without meandering lines irrespective of the speed at which the scanner is moved (the "scanning speed"). However, it is normally difficult for most users to move the scanner such that the rollers consistently contact the document. This commonly results in scanning document portions while the roller members are separated from the image. When this happens, the scanner to document distance ("scanning distance") cannot be obtained, accurate image scanning is therefore not possible, and linearity may be lost. As a result, the scanned text lines may meander or be scanned at an angle.
Another drawback to this type of scanner is that the overall size of the scanner wand housing (particularly the scanning window end) is rather bulky in relation to net scanning area to accommodate the rolling member and an encoder unit. The encoder unit generates a pulse signal according to the distance traveled in relation to detected rolling member rotations as the scanning wand is moved across the document. Durability-related problems also arise with frequent use due to the use of rollers or other mechanical moving members which frictionally engage the paper documents, which wear and can break down.
Two-line sensor handheld scanners eliminate the need for a roller member have likewise been developed. These two-line sensor scanners generally comprise two sensors (a first and a second sensor), and compares the image scanned at any given position by the first sensor, which precedes the second sensor in the scanning direction and therefore scans a given point first, with the image scanned by the second sensor to detect the scanning speed from the time difference between scanning the two images. An exemplary two-line sensor scanner apparatus is disclosed in U.S. patent application Ser. No. 08/499,537, filed Jul. 8, 1995, naming T. Nitta and M. Aoki as inventors and commonly assigned with the present application to Seiko Epson Corporation of Tokyo, Japan, which is incorporated herein fully by reference. As disclosed in that application, the two line sensor technique eliminates the need for the roller member, and thereby resolves various problems present in roller-type scanners.
As shown in FIG. 36, this two-line sensor scanner, in addition to two line sensors 101, 102 (referred to below as first sensor 101 and second sensor 102) as one-dimensional image sensors, two light transmission holes 111, 112 spaced apart at a prescribed interval, is configured such that the light from LED 1 passes through light transmission holes 111, 112 and irradiates characters, picture images, etc., presented on a source document. In turn, the reflected light is received by first sensor 101 and second sensor 102 via optical system 3.
FIG. 37 illustrates a block diagram of an image processing system employing the 2-line sensor scanner of FIG. 36, including the image input member 100 (scanner) and the image processing member 200 that processes the image data read by image input member 100. As discussed hereinabove, the image input member 100 comprises the first sensor 101, second sensor 102, read timing signal generation circuit 103 which provides read timing signals to these sensors, first and second buffers 104, 105 which store image data read by the first and second sensors 101, 102, input buffer switching circuit 106 which selects one or the other of first and second buffers 104, 105 for storing image data read by the first and second sensors 101, 102, and output buffer switching circuit 107 on the output side which fetches image data from the buffer of the first and second buffers 104, 105 in which a predetermined amount of image data has been stored and transfers it to image processing member 200.
This input buffer switching circuit 106 selects, for example, first buffer 104, whereby image data read by first and second sensors 101, 102 is stored in first buffer 104. When a predetermined amount of image data is stored in this first buffer 104, then second buffer 105 is selected or switched in and data read by first and second sensors 101, 102 is stored in this second buffer 105. Thereafter, when a predetermined amount of image data is stored, second buffer 105 is switched out from receiving image data and first buffer 104 is once again selected.
When a predetermined amount of image data is stored in first buffer 104 and the storing of image data switches to second buffer 105, for example, the buffer switching means 107 on the output side selects first buffer 104 according to a prescribed timing, fetches the image data stored in this first buffer 104 and transfers it to image processing member 200.
Image processing member 200 comprises mainly buffer 210, speed detector 220 and image correcting device 240. The speed detector 220 generally includes a first feature extractor 221, second feature extractor 222, feature buffer 223, feature comparator 224 and overall speed judgment device 225. Typically, the speed detector 220 components and the image correcting device comprise interdependent threads executing on one or more processing devices present within a general purpose computer.
In this type of configuration, image processing member 200 temporarily stores the image data read by image input member 100 in buffer 210, performs speed detection by means of speed detector 220 and corrects the image based on the detected speed by means of image corrector 240.
Typically (though not shown here) hand scanners normally include an input button as a switch for facilitating user coordination of image input. Only when this input button is depressed will the scanning wand 100 will perform scanning operations. Therefore, even if image input member 100 is moved, if the input button is not depressed, no image data is input. By contrast, even if the scanning wand 100 is not moved, if the input button is depressed, image data will normally be input and a stationary image will be retrieved.
Further, in image processing member 200, image data is fetched from buffer 210 and scanning speed is detected by speed detector 220. More particularly, first feature extractor 221 extracts the features of the image data from first sensor 101 at a certain point, that feature data is stored in feature buffer 223, and the feature data stored in this feature buffer 223 is compared by feature comparator 224 with the features extracted by second feature amount extractor 222 from the image data from second sensor 102 to check the conformity of both. This means determining if the input position of the image data from first sensor 101 has arrived at second sensor 102, and measuring the time lag before correspondence, if any, is reached. Therefore, using the interval between the two as a basis, speed judgment circuit 225 judges the scanning speed of image input member 100. Further, using this scanning speed as a basis, image correction means 240 corrects distortion in the input image data. This correction processing need only be performed on the image data input from one of the sensors 101, 102.
Image input devices which use the aforementioned 2-line sensor type scanning wand and associated image processing system perform quite well in correcting image distortion as a result of uneven scanning speed. However, when the user initiates wand movement, stops wand movement, and then restarts wand movement all while depressing the scanning input button (which commonly occurs as the user pauses to review what he has scanned or plans to scan next), this system has no means to consider the time period image data input scanning was stopped or the scan stop state. Therefore, even if the speed was temporarily reduced to zero, distortion correction would be performed based on an average speed sought from the distances traveled before and after stopping, without taking into account the time stopped. This results in an inaccurate scan.
This shortcoming is best explained in conjunction with Japanese language FIGS. 38A and 38B and similar English language FIGS. 54A and 54B. FIG. 38A shows the relationship between time and scanning speed for an example scan of the Japanese text "" (fukusosahoko) where the user momentarily stops moving the scanning wand 100 from time period t01 to t02 but continuing to depress the scanning input button. In the figure, points v1, v2, . . . , v7 represent the scanning speeds calculated by the speed judgment circuit 225 and actually indicate the average speeds in the intervals indicated by the arrows extending respectively therefrom.
Likewise, FIG. 54A depicts a scanning speed vs. time diagram for the scan of the English text "paragraph". In both examples, in the interval between time t01 and time t02, the scanning speed is actually 0, the aforementioned 2-line sensor image processing system would still consider the scanning wand as moving in the t01-t02 time interval at the average speed va. Subsequently, image correction is performed on these scanned images utilizing this false average scan velocity. The curve indicated by a dashed line in each of FIGS. 38A and 54A represents the estimated speed change in the interval between v5 and v6.
Therefore, even if the scanning speed in the interval between times t01 and t02 is actually 0, distortion correction is performed assuming that travel is still taking place (in fact, hatched area da represents the falsely assumed distance traveled by scanning wand 101 during the t01-t02 time interval), resulting in an overcorrected, distorted set of image data. The graphical result of this overcorrection may be represented as shown in FIGS. 38B (Japanese) and 54B (English) respectively, where the image collected by the second sensor 102 appears "stretched out" during time period t01-t02, as it takes a particularly long time to fetch the same image data. Also, though distortion correction is performed by image corrector 240 based on speed information (v1, v2, . . . , va, . . . , v7) calculated by speed judgment circuit 225, since distortion is corrected according to speed va during the interval travel is stopped as described above, a correction result can only be obtained in which distortion remains as in the "" (sa) part of "" (fukusosahoko) as shown in FIG. 39C, or the "a" in "paragraph" as shown in FIG. 55C.
Moreover, in using the 2 line sensor image processing system described above, a less-adept user may depress the input scanning button while still attempting to align the scanning wand with the beginning of the desired image to be scanned. In other words, it is common for the user to depress the image input button before the scanning wand is correctly positioned. Conversely, it is quite typical for a less-adept user to keep the image input button momentarily depressed once an image scan is completed before ultimately releasing it the input button is depressed even though scanning may be stopped. Either situation results in "over-scanning" in which unnecessary image data is unintentionally acquired and must be filtered out. This adds unnecessary memory and processing overhead and may result in a blackened, grayed or otherwise distorted image considered undesirable for character recognition or similar image operations.
It is, therefore an object of the present invention to obtain high quality images with negligible distortion through detecting a scan stop state during scanning by the scanning wand when scanning is temporarily stopped and then resumed while the scanning input button remains depressed throughout, thus facilitating distortion correction processing that considers a stopped state. It is also an object of the present invention to prevent disable correction processing when scanning motion is restarted after stopping for a short period with the input button depressed to filter out undesirable image data acquisition during wand alignment and pausing operations, and thereby improve memory and processing efficiency. It is yet another object of the present invention to recognize and disable scanning when the input button is switched OFF after the scanning wand has not moved for a at that position for a short period with said input button depressed, thus effectively utilizing memory resources and reducing correction processing time.